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THE IMPROVEMENT CF AGGREGATE TEACHING 
ErFECTI VENPSS IN A SChOOL DIVISION 



A'^STRACr 



This proposal maintains that, over a poriod of about five years, the 
aggregate ievel of teaching ef fuct i voness in a ichool division can be 
improved by careful attention to the administrative decisions of teacher 
selection, assignment, development, and retont ion/re I ease. With the 
e>:ception of assignment, good decision-making rn these areas is dependent 
upon a sound teacher evaluation scheme. 

Many such schemes have been proposed, and have been more or less 
Ineffective, partly bedause no sound research basis existed fur themi 
However, it Is now possible to construct a model, the four behavioral 
dimensions of which have been validated by empirical studies* The 
dimensions of warmth, indirectness, cognitive development and enthusiasm 
have been operationally defined, have bjen measured, have been shown to 
relate positively to desirable attitudes and/or achievement levels In 
students, and can be developed, in teachers, Additionally, at least two 
of them can reasonably be predlctc^d. Thus these dimensions allow 
predictive ovaluation for teacher selection, formative evaluation for 
teacher development, and summative evaluatfon for retention or release 
decisions. 



The fourth area of administrative decision-making, teacher assignment, 
IS less clear-cut. The match between teachers and assignments, based on 
training, experience^ and preference can probably be improved by consultation. 
The match between teachers and students -ased on congruent expectations of 
role behavior (the students' mutal expectations for teacher behavior 
matching the teacher's expectations for teacher behavior^ and vice versa) 
can probably be Improved by arranglnn for student selection of preforred 
teacherCs:, or by assassing expectations by a survey Instrument, However, 
the gains here are hypothetical. Ar yet a sounder notion of constructive 
mismatch for growth along certain iohavioral dimensions Is also at the 
deve lopmenta I stage. 

Given appropriate data, and commitment to improving the quality of 
administrative decf sion-maki ng in four vital areas, aggregate teaching 
effectiveness can be Improved to the great advantage of the students in 
a school division* 



The Improvement of teaching ef fecti vaness of a school division, 
In the aggpegate Is, it will be maintained, B long term projict re- 
quiring commitment over a period of about 3 years. This proposal 
Involves four related activities each of which contributes to the 
Increasing of the aggragat.e ef fecti ver.ess of the teaching staff. The 
first activity, teacher selsction, contributes to aggregate effectlvene 
by attempting to eliminate teacher candidates who would be Ineffective 
In practice. The second activity, teacher assignment, attempts to 
Improve aggregate effectiveness by Improving the match between teachers 
and assignments. Teacher development, through self-help techniques 
and more formalized in-service processes, is an Important contributor 
to staff effectiveness, at least potentially. The final element or 
activity Is also a significant contributor to aggregate effectiveness, 
although not commonly considered as such. That is the release of 
teachers who have demonstrated to be relatively frieffectlve, and for 
whom the self-help and fonnat In-service programs have not been useful. 
It Is maintained that only such a comprehensive plan, persevered with 
over a period of years, can significantly affect the level of teacher 
effectiveness in a school division. 

One Important reservation should be entered here. All of the 
suggestions to be made here have been demonstrably effective In other 
contexts, and used for different purposes. But most of the activities, 
techniques, and Instruments described here have never been used for the 
purposes suggested here. This does to some extent reduce the amount of 



confidencG ona C3n hav3 in iho prob?)bla outcome. For example, tho 
rolatlonship oGtwoon the verbal facility togt scores of teachena 
and pupil ^chiGvomQnt has boon acmonstrntod sovora ' timQs, Yot 
several researchers have cau+ionod agninst ua^ing this domonstrated 
relationship, which ■ ; net necessarily -jjsjI, for administrative 
purposes. However, the probnblo cost-Qf focti vonoss of solocting 
teachers on this b'lsis is Gxtremoly high comparod to some other 
bases which have been used in the past. Thus the benefits seem to 
exceed the risks, in this particular cdso, and this particular 
technique wM I be proposed for adm i n t strat i vo uses. Given the general 
resc-rvatlon then, that mony of the suggestions made her© are basec 
on relatively recenr research findings, and are hence untried for 
the kinds of uses proposod, the suggestions made here sesm well worth 
the consideration of practicing administrators, ©specially when it is 
considered that prosont teacher evaluation, and hence effectiveness, 
schemes have been very unproductive. 

The arrangeinent of the paper Is as followsi some genoral 
questions raised by the notion of of feet i venoss In toachlng will be 
considerod brief Iv, and then a general toachor ovaluetion mode! will 
be described. The four dimGnslons of the model and their application 
are then analyzed in some dotal!. The next section deals with the 
question of teacher assignment, which is romowhat different from the 
issues of selection, doveloprnGnt, and relepse In that evaluation is 
not of fundamental importance. The final two sections suggest some of 
the problems of implementing such a proposal, and draw some conclusions 
and further fmpMcations from the material already presentod. 



Towards More Fff» r:tlTO Teachmq 

Any attempt to mprovs the offoctivenoss of toachlnj must 
recognize tho rol.tlve K,ck of succeed of m,ny thousonds of such 
attempts which have boon ™ade In the p==t. C.u+ior and precision 
are hence e.ssntlal, a„d any proposals must bo rolatlvoly tentative 
and can only 5, Justified ty the i.portanco and urgency of the task. 
In this proposal teachihg ,s defined as "the e.ertlon of the behavioral 
influence... <G,go, ,972.. p. 43, Obviously behavioral l„,|uo„,e can 
be positive or negative, that Is it can change behavior in desired 
directions or non-doslred directions, and It c,n also be more or less 
effective, that Is It can make large scale or small scale changes. 
More effective teaching then Is defined as the exertion of behavioral 
influonce In desired directions and In larger measure than before. 

The ways in which this Influence is e.erted on students are, 
es Will be described In detail later, through various .teacher behaviors ' 
in the Classroom. Thus the notion that teaching can bec^e more effective 
assumes that the behavior of teachers In classro^s can be changed. This 
is part of a general assumption about human behavior which Is somewhat 
-popular currently, ,„ , ,3ce„t article i„ a popular magazine, entitled 
"Human Beings Are Not Very Easy to Change After All, An Unjoyful Message 
end its Implications for Social Programs,.. Et.ionI suggests that social 

°- core assumption that man can be 
taught almost anything and guito readily.., ,,972, p, 45) However, there 



IS B substantial body of rosonrch which domonstrg+es thnt teacher 



behavior can in fact bo changa^d, at least In the contoxt of formal 
teachGr training programs. For cxampiG, Turner (1963) demonstrated 
that the improvement of tuocher problom solving through training was 
refiOcted in the Incroasus In anhiovoment of pupils over porlods or 
several years. More generally^ a whole sorigs of micro-teaching 
activities have ricmonstrijted that teacher cl^issroom behavior can be 
changed, and the changos resulting are fairly stable. (See Berliner, 



A second mjor essumption being made here Is that enough is 
known about the dimensions of nffactlvD teacher behavior to enable us 
to decide what represents more effective behovior, There is some real 
doubt about th i s , 

-The value of most teacher effectiveness studies In the past- 
^has been limited by their attempts to predict teaching success 
di^rectly from assessments of the personal characteristics of 
teachers without considering any Intervening variable, by their 
unreliable assessment of rather gross features of classroom 
behavior^ pnd by their lack of attention to the varied contexts 
in which teachers work.., the questions to which investigators 
were confidently seeking answers half a century .^go can now be 
seen as answerable only on the basis of a great deal of research 
which has hardly begun. (Morrison & Mclntyre, 1959: p.. 41) 

However, recent research seems to Justify mere optimism. The most 

recent review of the research on ■'teacher effectiveness** in the 

Encyclopedia of Educational Research suggests that 

the resGBrch which is reviewed herein permits cautious optimism 
and indicates that the tools long needed for the analysis of the 
teaching- 1 earning process are gradually being developed* This 
optimism is In contrast with the conclusions reached in past 
rt^views, (Flanders -S 3imon, 1969: p. M23) 



The tontnt I voness of the conclusions nrrlvcid at so for In 
research on teacher effectiveness, and tho lack of clarity about their 
applicability to specif *c Gchool division situations, suggests the 
need for caution* Howover, the practical problem of ovaluatfng teachers 
Is so pressing that some uses of the research findings, however un^ 
certain and tentative^ seem essential, Thore is for the first time a 
reasonably well established set of resGarch findings on teacher 
effectiveness, they do offer some possibility of use in practice, and 
the main purpose of the paper is to describe briefly these findings, 
and suggest how they might be used. 

The dimensions of teacher behavior used hero were deycsloped 

originally by Gage, In his recent survey of research on teacher 

effectiveness. By surveying empiricol research, ho tentatively 

identified four dimensions of teacher behavior which seemed desirable. 

His procedure was to 

Present a series of operational definitions of teacher 
behaviors that seem, more or less, to belong in the same 
dimension. These definitions will bo drawn from various 
research procedures and measurl ng - i nstruments . Then I 
shall cite some of the evldince on which It Is possible 
to base the inference that these behaviors or characteristics 
are desirablo, (Gage, 1972: p. 34) 

The four dimensions he identified ore "warmth'-, "I ndf rectness*^ 

"cognitive organization", and ^'enthusiasm". As Gage points out, 

there Is nothing very startling about these four variables. They do 
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■not exhaust tho possibilities and they are roally only representa+i vg 
of tha things th^t resuarch on tooch i ng cnn presently support. What 
is Important about these dimensions "is their basis in Qmpirical 
resoarch". The mm with which others hove told us such truths in 
the past is matchod by tho i r untrustworth i ness . " (p. 39) To su-nmarl za 
the reasons for selocting these dimensions in vary briof farni, first, 
they are based on empirical rGsoarch; socond, raliablo instrumentation 
for measuring these toacher bohavlors genorally exists; third, the 
desirability of those teachers'' behaviors hns been demonstrated; and 
fourth, these behaviors can in fact be learned by teachers. 

The approach has of course some major defects as well. It is 
for from ccmprehGns i ve, and ignores a great many significant l<inds 
of evaluations of school effectiveness. However, recent attempts at 
global evaluation systems, and particularly those based on student 
achievement, have all been to soma extent unproductive. Tymin (1970) 
in an extensive examination of the Issue of ovaluating the of f ecti voness 
of education, is as posslmistic as he is thorough. He provides a series 
of questions which remain to be answered before a satisfactory model of 
evalyatlon can be constructed, (given in abbreviated forni): 

"I. Whose goals should be taken as the goals of education in any 
system? 

2. Assuming one knows somehow whose goals should be taken, how 
does one discover these goals? 



3. How does one decldo what outcomes shall be taken as proof 
of the accompli shmont or failuro to achieve those agreed 
upon goals? ° 

4. How does one measure those outcomos? 

5. How does one assess the contributions of various fnctors to 
the achievemonts so measured assuming one con measure? 

6. How does one relate behavior and measures at the end of school 
careers to conduct and achievement in adult life? 

7. How does one estimate the economic costs and the econonlc qains 
of various input and outcomes of the educational proems? 

^' clan?t?v/^H such analytically disparate outcomas as 

cogn ti ve adequacy, emotional well-being, creativity readiness 
for inter group life, ability to function In a d^ra^ 
ofmlocletyf"-'' appreciation of the cu I turar traditions 

9. Against what standards does one canpare a school's "achievements?" 

The validity of these observations is perhaps attested to by a number 
of recent failures to provide useful evaluations of the work of teachers. 

For example, the limitations of using standardized tests of pupil 
achievement for making decisions about teachers have been emphasized by 
recent concerns with perfortnanca contracting. The difficulties of using 
standardized tests to provide data for definitive Judgments about the 
effectiveness of performance contracting and the rewards due the con- 
tractors (Klein, 1971) are effectively the same as those Involved In 
making Judgments of teachers, 

Another interesting recent failure to measure teacher competence 
in terms of student achievement was based on the teaching of technical 



skills, which seem readily moisurnbla oufcwrnws, and used "performance 
tests". (Sea Kloin and Alkin, \912 for a discussion). Performancu 
objectivos were careful ly dofinQd, 10 hours of instruction were give-n 
by two groups of people, oxparienced teachors nnd non-toachers, and 
the outconiQ In terms of class nchiovemont was measurod. The results 
did not discriminate betwoen the sxpcrienced teachers and the non- 
teachers. This approach presutnably than could not In any way assist in 
discriminating degrees of taacher compotonco, sineo it cannot even 
distinguish between exporipncod and inoxporiGnced teachers. (Popham, 1968) 

The surprise or disappol ntmont frequently expressed in this 
connection is itself remarkable, since a relatively early and Gxtremoly 
thorough study of the characteristics of teachers concluded that 

Product measurements (estimBtos of the behavior or achievements 
of the pupils of teachers) have been widely acclaimed as 
desirable criterion data, but have been Infrsquently used in 
the study of teaciier behavior. Actually, the seeming relevance 
and appropriateness of the measuremont of pupil behaviors 
and their products as indicators of teacher perfonnance may 
be more apparent than real, for the producers of (or contributors 
to) pupil behavior or pupil achlovoment are numerous, and it is 
difficult to designate and parcel out tho contribution to a 
particular "product" mado by a specified aspect of the producing 
situation, such as the teachers. (Ryans, I960: p. 375) 

This is of course virtually the Identical conclusion arrived at after 

a great deal of discussion and debate with regard to the Coleman Report. 

Mood suggests that "the present rudimentary state of our quantitiatl ve 

models does not permit us to disentangle the effects of home, school, 

and peers on student achievement," (1970: p. 6) Specif icaMy with 



refQrencQ to toachors, Mooci goos „n to soy "we can only make the 
not vary useful obs.rvotion th«t ot tho present .o^ent wo cannot .ako 
any sort of mooningful quantltntive estlmato of tho offGct of ttaehers 
on student ach I avemQnt. " (p. 7) 

If it Is gmnted tho1 tha global svaluation schames had turned 
. out to be lass then satisfactory, then it sooms reasonable to base 
evaluation of teachers on so'Tiewhet different measures. One way of 
interpreting the enopmous quantity of research on teacher effectiveness 
since the last 30 or 40 years is that it too demonstrates the necessity 
for preilslon in evaluating teachers. "Where the earlier efforts made 
much use Of g'lobal ratings, the present day work relies much .ore on 
reliable counts of specific behaviors". (Gage, 1972: p, 206) Hence 
what will be proposed hera is a fairly specific set of dimensions, on 
the basis of which teachers can bo evaluated directly, (n due course 
as some Of the difficulties with us i ng. standard i zed scores of student 
achievement are elimfnated It may bo possible to add that dimension 
to the teacher evaluation scheme proposed here for school divisions; 
at present, however, student achievement scores seem not to pTOvIde a 
realistic means of evaluating teachers in the context of school divisions 
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Evaluating Teacher Ef focti vonoss in a Schocl Pi vision 

The current status of tsachor ovaluatton programs fn targe 
school systoms in the Unitsd S+ates w^s roported recsntly by the 
Educational Research Sorvice of the Africrican Aisociation of School 
Administrators (1972), Based on a swrvey of school systems In tho 
United Statos enrolling 25,000 or more pupils, and on returns from 
155 systems, the survey showed that the four major purposes of 
evaluations are "to stimulate Improvomont of teacher performance", "to 
decide on rpappoi ntment of probationary teochars", "to recommend 
probationary toachcrs for ponmonont status", and "to establish evidence 
wher« dismissal from sorvico is fin tssuo". To a question on the status 
of teacher ovaluation procodures In 1971-1972, (155 answered). 
110 systems noted that thoir program will remain unchanged, 35 noted 
that their program will bo rovised in 1971-72, 6 suggostod that they 
did not have a progrnm at prosont but would initlato one fn the 1971- 
1972 school yoar, and A suggootod ttiat they had no plans for a teacher 
evaluation program. To a question rogarding who was rosponslble for 
ovaluating (I OB onswarod), 77 of the rosponding systems noted that 
the principal was thrs solo ev.jluator rQsponslblo. In the overwhelming 
majority of school systems, clasflroom obsorvations of teachers by 
prfnclpnls or supervisors nro the standard method of ovaluating. In a 
majority of casos, a chaek list or rating form Is uaod. On the bails 
Of this syrvay, It Sfaems r^nsonablo to conclude that most Inrgo sehool 
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systems rn the United States still make uso of evaluation schemes 
the basis for which has been rather thoroughly discredited over a 
substantial perfod of time. Morrison and Mclntyre summarize the case 
against rating scales thus: 

Oesplto their popularity several objections can be raised against 
rating scales. One of thoir more serious limitations when used 
for assessing the classroom behavior of teachers is that an 
•xtenslvs amount of information about what has gone on has to 
be reduced to subjective and Impressionistic endorsements on 
a few scales. Since they are heavily dependent upon the subjective 
impressions formed by the individual rather than their re! i ability 
from one occasion of rating to another by the same rater, or botweeen 
two or more raters on the same occasion, Is highly variable. Also 
when the rater is presented with several supposedly distinct character- 
sties to assess he may in fact be unable to distinguish between them, 
leading to a tendency to rate an Individual as 'high', 'average» 
or low* on most of them. Finally the Infonnatlon available to 
the rater can vary very much from one characteristic to another and 
from one Individual to another. (1969! p. 22) 

This Is not to say that rstlms of toochers are nGcessarily and invariably 
inaccurate. However, unless the sot of guidelines proposed by Ryans ( I960! 
p. 75) or something simllap Is observed, those ratings will not be very 
reliable. The conditions are not nonnally observed In actual teachor 
rating systems, as 'the ERS Survey shows. The rating technique has never 
been acceptable to teachers, who havo strongly resisted, via their pro- 
fesstonBl associations, the suggostlons that either their salary or their 
Job security should bo determined by ratings. ThoIr success In achtevlng 
Job tenure and pay scales unaffoctod by Judgmonts of their competence 
arrived at through classroom visitations Is clear evidenco of the un- 
spoken agpeafflsnt by vlrtuoMy all concopnod that classroom visitation and 



ERIC 



- 12 - 



rating Is netther a roliablo nor a valid mGans of evatuatlng tiiacherSi 

The model proposed here satisfies the definition of educatfonal 

Qvaluatfon adoptod by the Center for the Study of Evaluation at UCLA 

which suggests that educational evaluation Is 

the process of determining the kinds of decisions that have 
to be made- selecting, coMecting, and analyiing information 
needed in making these decisions; ond then reporting this 
Infomatlcn to approprlato decl s Ion-makers. (Klai n/ 1 97 h p*9) 

The three kinds of decisions which the model proposed here provides 

Inforrnatlon for armi teacher selection, teacher development, and teacher 

release* Different kinds of Infonnatlon are required In each of these 

different decision areas, and thus the model provides for predictive, 

fonnative, and summative ovaluatlon. 

The model Is summarized In tho following chart, and subsequent 

sections of the paper elaborate on the four dimensions of teacher 

behavior used In the modeh 

THE EVALUATION OF TEACHER EFFECTIVENESS: 

AN ADMINISTRATIVE MODEL 



Dimonslons of 
Teacher Behavior 

WARMTH 
INDIRECTNESS 
COGNITIVE DEV. 
ENTHUSIASM 



Types of 
Evaluation Yielded 

Predict I ve 
Format I ve 

Format i ve 
SummatI ve 

Predictive 
Summative 

Formoti vs 
SummatI vQ 



Decision Area 

So I Oct I on 
Development 

Development 
Rotent t on/Re I ease 

Selection 
Re I ©asQ 

D©vol©pm©n+ 
Rotent I on/Re I ease 
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Tho dimension which Gago icibois "warmth" has boon identified by 
three different insTrumonts, tho Minnesota Toachor Attitude Inventory, 
the Canfornia Scale, and the TQachor Characteristics Schoduio, and 
these three Instruments correiate folrly closaty. rurthennore, all of 
the instruments identify attitudos and hehciviors which correlate posltivoly 
with favorablo assessments of the toBChors by both students and objective 
observers, and with the achievement of studonts. As Gago summarizes the 
s Ituatlon, 

A substantial body of ovidonco supports two conclusions: 
a) teachers differ roliably from one another on a series 
of moasuring instrumants that seem to havrg a great deal 
in common, b) thoso rsllable Individual differences among 
teachers aro fairly consistently related to various dosirablo 
things about teachers, (Gag©, 1972: d, 35) 

If tho dosirabi lllty of warmth Is accaptad, and It seems Incontpovertfbh 
the various instryments described here con then be used prod let Ivelyj that 
Is as selection devices In hiring teachers, with roasonable probability 
that the toachers with favornblu scores will be effective toachers, both 
In terms of studonts attitudos towards them, and the achievements of 
students, 

In oddltlon, there are further possibilities Inherent In these 
pesearch findings on the dlmenaion of warmth. Slnco the M.T.A.U has • 
been shown to relate quite closely to favorable ratings of the toachers 
by their pupils (Yoss, 19675, and sine© student ratings of teachers have 



biion shown to bo extrorneiy useful in changing toHchor behavior, this 
dimension can also provldo fonn;itIvfc' ejva I uation , 

.Student ratinas of t-jachurs "hove yielded usoful ovidenco on 
tGaching at IqvgIs as low as fourth grada snd as hiqh ns medical 
school". (Gage, IQ72- p. 172) In a long strias of oxperlments in 
feodbfflck from studont teachars, a number of rosoarchors hav© demonstrated 
that studont foadback, porticulBrly In the form of wrltton ratings, is a 
roll able and valid way of hoi ping tonf:hcrs inprovo their toachlng, 
at least as porcGived by students. Tho improvement here doos not 
genera My reflect in gains in student ach iDVemant, but only In more 
posit ivs attitudos in studunts. This technique, It should bo emphasized, 
is really a private transaction botwean students and their teachers, and 
has no validity whatsoevor for ndmi n i stroti vo uses, In for oxampio 
evaluating toachors for retention or rolonse. NHv^rtholoss, It Is a 
form of ts.icher devQlopmont which has had good effect in fi number of 
different contexts end could well bo an Important element In in-service 
training of teachers, provided tht' limitation on its use Is clearly 
spel led out, 

Tfiu Pi mans ion of " I nc) i roetness " 

Gage bnsos this dimension on two dlfforont but rolntlvely closely 
rolatod research areas. The first Is usual ly idontlflQd as "Intaractlon 
anolysiB", and is associated with f-landers. One characteristic of teacher 
verbal behavior has been found in g|ofo nBsocIntion with both constructlVQ 
studont nttttydoB and favorable studont achlfivcmont levels, Ins number 
of studies by different rosoorchers at different grade lavals, ond In 
dlfforont pnrts of the United States. 



The parcentage of teachor statements that make us© of ideas and 
opinions previously expressed Dy pupils Is directly related to 
average class scores on attitude scales of toacher af feet I veness^ 
liking tht class^ etc. as well as to averag© achievement scores 
adjusted by Initial abiiity, (Flanders ^. Simon, \9n9: p, 1426) 

The variety of studies, and of ways In which the interaction analysis data 

has been.ustd, m revlew^^d by Flanders and Simon, suggests that such data 

enables one to predict the relative succesB of ^eachers who fall Into 

high or low categories In some types of verbal bohavlor^ to train teachers 

to exhibit the apparently desirable verbal behavior, and to distinguish 

classrooms in which achlevoment and attitudes of students will be relatively 

poor* Thus this dimension has predictive, formative, and summative, and 

consequently this dimension would be used to develop in-service training 

programs for teacher deveiopment, and in fivaluatlng teachers for retention 

or release. 

The Dimension of ^'Cognitive Deveiopment' ' 

This cemenslon of effective teacher behavior Is certainly the least 
understood and least lmmedI«toly useful way of measuring teacher effect! venesSi 
Since it has not yet been satisfactorily defined operationally, It will be 
necessary here to suggest the use of a number of proxies, or Indfcatorsi 
in place of cognitive development directly measured by some reliable and 
valid Dbservations or a test instrument. Unfortunately, these proxies are 
not "process variables*^ or oven clearly related to such variables, and 
consequently do require the Inference that the characteristics measured do 
affect toachers^ classroom behavior, Thoy have all been shown to correlate 
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positively with student achievement levals; what is in question is the 
degree of ciusai relationship. Proposed are first, verbal fscility; 
second, academic achievemsnt. and third, recency of academic training. 
Each of these prox-ies requires some description and specification. 

The importance of the teacher's verbal ability was clearly 
demonstrated by the Coleman Report. 

The most significant school service variable In eKplainlng 
student achievement (measured by a vocabulary test) was a' 
teacher characteristic, the teacher's verba I I f rty, As 
with the other findings of this nature that we have discussed 
care must be used in interpreting the meaning of such results 
...However, if one views teacher's verba! ability as the proxy 
measure for a number of related skills and qualities, the 
Coleman Report finding can be Interpreted In a meBnlngful 
fashion. If the measure of verbal ability is taken to 
represent the general intelligence level of the teacher, 
the finding can be construed to mean that an intellectually 
fascile instructor is moro adopt at tasks such as finding 
means to motivate students, adapting materials to their 
ability levels, and communicating in ways which make the 
subject matter more understandable. This Is an Interpre- 
tation which IS totally consistent with the observations 
and conventional wisdom of untold thousands who have them- 
selves been teachers or who have supervised teachers. 
Guthrie, (1970: p. 37) - - - 

Although the relationship between teacher verbal ability and 
student achiovemant Is not currently questioned, since the findings 
of the Coltman Report, the usefulness of this measure of teachers for 
administrative purposes Ig certainly debatable. Thus for instance 
Gags points out that this relationship Is correlational, but not 
necessarily causal, Consoquontiy, "wo cannot proceed to Improve studtnt 
achievement by hiring teachers with graater verbal ability". (1572i p. 33) 



Similarly, Mood points out that: 



If w© went about increasing the verbal ability of toachors 
the increase that might result in student achievement would' 
bo far less than what would be calculated by using the 
equation that rslatss to achievamsnt. The reason Is that a 
specific increase in verbal ability would probably not be 
accompanied/ by a corresponding increase in all the ether 
attributes that verbal ability is serving as a proxy for. 

( IV/U! p, 3) / 

However, other writers disagreQ. For mmpla, Levin suggests ' that • 

Recruiting and retaining teachers with higher verb 
five to ten times as effoctiv© per dol'ar of teach- 
in raising achievement scores of students as the strategy of 
Obtaining teachers with more Qxperlenca. (1970.- p. 24) 



a I scores Is 
er expenditure 



The relationship between academic achievement of teachers and the 
achievement of students has been shown by many different studies at 
various times, and will not be further described. However, the finding 
that recency of academic training is more closoly associated with student 
achievement than the ovorall level of training obtained is relatively new, 
and is an outcome of a ro-analysis of Coleman Report data. Hanushek 
found that 

Recent educational experiences - olther undergraduate or graduate 
level - are important. Thus efforts to have teachers returned to 
school during summers seem Justified In terms of effects on 
education. The cumulatlvo effect (the master's degree or total 
units; IS not as Ifflportant as recant Involvement. (1970! p. 92) 

In summary, the dimension of ''cognitive development" In teachers 

would be assessed by the development of three proxies, each of which 

has been shown to be associated with student achievement. It seems 

roasonable to Infer that teachers with hlcjh relative scores on those 
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proxies would then be ralativsly more effective +han othar teachers. 
It is suggested that the administrative use made of these proxies would 
be both predictive and summative. That is, at the teacher selection 
phase teachers would be so I acted In part on the basis of verbal ability 
scores, academic achievement, and recency of academic training. In 
the evaluation for ratention or releaso phase, similar use would be made 
of the proxy scores. Since these scores do not lend themselves very 
readily to the developmsnt of in-sorvice training programs, they have 
no formative usefulness. 

The second proxy, ability to explain, arises out of a series of 
experiments and studies by Gage and colleagues based on the notion that 
teaching behavior consists of a group of technical skills, amongst other 
things, and that on© such skill was the ability to explain. Studies of 
this ability in teachers showed that the differential effectiveness of 
teachers as explainers was perceptible to trained observers using a video 
and audio record of the explanations given by the teacher in the classroom. 
More Interestingly, from the point of view of teacher development, a 
mangal developed by Miltz on "how to oxplain" which was based on the 
previous research, increased the ability of teacher trainees to explain. 

The Dimension of "Enthusiasm" 

The distinction Ijetween "snthus i astic" or "stimulating" teaching 
and "indifferent" teaching has been used in a substantial number of 
resiBrch studies. In a rsgont review of these studios Rosonshlne (1970) 
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distinguished between high inference studies, which "require considerable 

inferring from what is seen or heard in the classroan to the labelling of' 

the behavior" (p. 500) and low inference studies In which specific behaviors 

are carefully identlfiod. The six high Inference studies reviewed 

provide strll^ingly consistent results. They suggest that one 
Of the patterns of effective teaching behavior indent If led by 
Ryans (1950), namely Pattern ?, described as "'.Hmw^atinq 
imaginative, surgent versus dull, routine teocher tehavlor" Is 
significantly related to pupil ach lovement. (p. 506? 

Reviewing the low infsrenco studies which are particularly use.-jl for 

teacher training programs because they do give some indication or the 

specific behaviors which teachors should exhibit, Rosenshine Identified^ 

a number of behaviors as components of "enthusiasm", and hence desirable: 

The teacher who scored high on the Energy factor appears to exhibit 
three types of related behavior. Firsf, ha is energetic, a rapid 
speaker, mobile and enthusiastic, but relaxed. Second, he asks 
varied questions, emphasizing questions of interpretation and 
opmion^as well as factual questions. Third, he praises frequently. 

Clearly, the high inference studies above ralate more generally to 
the summatlve evaluation in the evaluation model here and the low Inferenco 
studies relate to the formative evaluation stage. There seems little doubt 
however that the same general teacher behavior is being noted in both types 
of studies. Rosenshlnefs final summary of the studies on enthusiBsm 
suggests that: 

The results of high Inference studies provide evidence that ratings 
given to teachers on such behaviors as "stimulating", "onergetic"^ 
"mobile" "enthusiastic", and "animated" are relatSd to measyres of 
pupi I achievement. The results of low inference studies suggest that 
the frequencies of such variables as movement, gesture, variation In 
voic©, and eye contact are related to pupil achievement, (p. 510) 
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There is another rQsoarch area whiGh seems fairly closeiy related. 

Presumably enthusiastic and energetic teachers have differential Impact 

on the attention level of the students. As Rosenshine points out, the 

result o.f the study described "may occur becausa animated behavior arouses 

the attending behavior of pupils", (p. 510) and in fact the ettentlon level 

of pupils is Itself an area of investigation with a long history. This has 

been reviewed and summarized by Jacksoni 

What, then^ are the chief conclusions to be derived from 

systematic studies of classroom attention, starting from the 

early work of Morrison and extonding to the most recent 

reports? First^ although the amount of attention may vary 

considerably from class to class and even from minute to 

minute within a class, It would seem that most of the time 

most students are attending to the content of the lesson. 

Second, the. amount of attentfon In the classroom is often 

less than meets the eye,.. Third, the amount of attention 

even when crudely estimated by an outside observer, seems 

to be significantly related to other educational variables, 

such as scores on achievement tests and estimates- of teacher 

effectiveness. There Is also the suggestion that the amount 

of" attention may not be closely related to the students^ • , 

intellectual ability. In sum, those conclusions provide ample 

Justification for further study and speculation, (I968f p. 102) 

Given the existence of the relationship between enthusiastic teaching and 
the attention of pupils which is suggested by the research, and which 
certainly seems consistent with common sense and the experience of 
practitioners, the use of obsorver estimates of the attention of the 
class to the teacher does not seem an unreasonable evaluation technique. 
Furthermore, 

An estimate of pupil attention is commonly used by teachers to 
Judge their personal effectiveness In the classroom. The 
possibility of massive inattention, signaling the loss of the 
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teacher s authority, is frequontly reported as a dominant fear 
among beginning teachers. Second, students also worry at times 
about their inabMity to remain focused on the task- at hand. 
Boredom IS one of the chief complaints of students who ars 
having difficulty with school . (Jackson, 1968: p. (02) 

Thus there seams to bo substantial justification for the use of observation 

of pupil attention as an ele,,ient in formative evaluation as well. If the 

anKfeties of teachers, particularly beginning teacher's, and the boredom 

of students can be somewhat relieved by training teachers to be more 

enthusiastic or energetic, as the research suggests is possible, this 

could be a most useful dimension of tQachsr effectiveness. 

Teacher As signmBnt; Improving tho Match 

The general issue of match or consistency between teachers and their^ 
work assignments can be considered in three different wBys. First, the 
relatively simple question of the wishes of the teacher with regard to a 
teaching assignment; second, the more compIeK question of role expectations 
and role conflicts for teachers and students and third, the even more 
difficult, and relatively unexplored question of what is the most productive 
match (or mismatch) between students and teachers for promoting student 
growth along psychological, social, or cognitive dimensions can be considered, 

A recent study of the teacher workforce in British Columbia (Wallin, 
1971) showed that only 57.47^ of the teachers responding (n=l6,387) 
expressed satisfaction with their teaching assignment. The study built up 
a "quality of assignment" Index, of which the negative, or m I sass igned 
category, included a teacher who reported that he (she) 
"(I) was dissatisfied with his present position, 
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(2) found nono or few formal courses in his pre-service 
education to be relevant to his current assignment, 

(3) indicated that his training assignment did not match, and 

(4) has not participated In any non^formal credit courses that 
pertain to h i s present assignment." 

The positive or wel l^asslgnod categories included the teacher who reported 
that he 

"(I) wanted to have his present assignment next year, was 
satisflQd with his assignment, 

(2) has had relevant formal courses in, his pre-servlce education, or 

(3) has taken non-credit type courses and programs related to 
his present assignment," 

The study found that perceived misass ignmont was relatively common, with 
12.661 of teachers falling Into this category, and that teachers who 
felt well-assigned Included 26. 46^? of the workforce. Misass ignment seemed 
to be particularly common amongst rather welHtrained toachers. 

This data on assignment has two different kinds of significance - 
first, it seems to indicate a concern amongst teachers regarding the misuse 
of talents, abilities, or training, and second, it probably also illustrates 
a resentment over lack of control of work, which Is common ai^ongst pro- 
fessionals working in organizations, including teachers. (Corwin, 1970; 
p. 46) Thus a reduction of this misass ignment requires two kinds of 
solutions, a process of consultation, to Increase the sense of control, 
and also a serious attempt by administrators to reduce mismatch between 
desired and actual assignments, and hence to alleviate concerns about the' 
misuse of talents, abilities, or training, 
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Another version of matching can be based on the research on role 
behavior, particularly in organizational contexts. The theoretical model 
of factors in role conflict and ambiguity used here (see diagrarM) Is 
that of Kahn et : I , (I 964) . 

ROLE SENDERS _ foCAL PERSON 



Experience 



Response 



Role Role 
Expectations Pressures 



I 



Experiance 



Response 



Psychological Coping Efforts 
Conflict" 



The model Is based on the notion of a role episode, "a complete 
cycle of role sending, response by the focal person, and the effects of 
that response on the role senders." (p. 26) In practice, conflict and 
ambiguity Is experienced by the focal person through a series of such 
episodes. The notion of matching applied here suggests that appropriate 
teacher assignment might be able to minimize role conflict by ensuring 
a relatively high degree of congruence batweon student expectations 
and teacher expectations of appropriate role behavior. One caution Is 
In order: although studios of expectations are common, studios of teacher- 
role performance are rolativaly uncommon, and it Is quite possible that 
the continuity between role expectations and role pel^formance will not 
be direct and straightforward, so that knowledge about expectations will 
not necessarily yield knowledge about classroom behavior. However, It 
seems possible that If there Is a good match between student and teacher 
expectationi, that those expectations might more readily b© translated 
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Int- the w><p.ct.d boh^vior, F,r , thorc^h review of th^ sociological 

research n rolu theory,. os It hos bcon utilized in rosoarch on toochors 

and teaching, s^o Piddlu (1969), 

In order to achieve congruence In nssignment, prior knowledge 

regarding the ©Kpectations of the teacher and the students Is necessary. 
Although a good many test instruments for ascertaining role expectations 
of teachers and student exist, (scg e.g, those used by Bogen, 1954^ and 
Cheong and DeVault, ( 19^:6) It would probably be necessary to develop a 
simple test for the purpose, 

MnothCT tochntquo, which might be used together with or Instead of 
the psnci l-and-paper test of oxpectat tons. Is simply to offer students a 
choice of teachors and assum^i that they possess enough knowledge to choose 
th© teachor whose behavior will conflict least with their expectations. 
The obvious and common-sense reaction amongst educators to this suggestion 
is that the students would all wish to be taught by the least demanding 
teacher. However, as is often thiS case, research does not support the 
glib genQra I i zations of common sense. Both American and British studies 
have shown that thsre Is a high degree of unanimity In what children 
regard as desirable qua If ties of teachers. Additionally, "children, 
especially younger children, described 'the good teacher* more in terms 
of his teaching than did teachers or student toachorsi the latter group 
in particular emphasized the personal aualltles of teachers". (Morrison 
L Met n tyre, 1969: p. 109) The same British study being quoted above 
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Wideagreemont among pupils about tha importance of firmness, 
justicQ, tho avoidance of corporal punishment, friend linoss, ' 
knowiadge ^nd, most of all, participation in class actlvitisg. 
Amongst Junior school pupils, oncouragemant to work hard was 
stressed, as was poiltoness by the girls; and amongst socondary 
school pupils, cheerfulness and oxploininq the work was emphasized 
(f-tomson Mdntyre, 1 959 • ■ p. 109) 

These studies load one to expect that students might be quite capable of 
choosing tho toanhar who most appropriately satisfied their expectations 
through his teaching behavior, and Indeed trials of this in other areas, 
which generally hava been unsystematic and not reported in the research 
literature, have found that pupils did Indood derive some satisfaction 
from the ciioice of teachers. Just as umivorsity students do. At the 
university leva I, studies of student ratings of teachers have found that 
the fpct that the course is elective, In other words that students chose 
a particular teachers, has a significant Impact on the ratings of the 
teacher by the students. (Gage, 1972 p. 171) 

The notion of matching dsvalopad by Hunt and his colleagues might 
more accurately be described as constructive mismatch. They reject 
simple matching in terms of personality: 

We. .. disagree with some prevalent views of education, especially 
at the college level, which emphasize placing the student in the 
environment that is most congruent with his existing personality 
structure. In our view such procedures simply pranote arrestation 
and thereby defeat the process of growth and progression, which 
should be the major goal of education, (harvey. Hunt, & Schroeder. 
lyoM p. 340) . 

Rather, their proposals rest on a value assertion- "abstract, conceptual 
structure and Its associated characteristics of creativity, flexibility, 
stress tolerance, and broad-spectrum coping power is a desirable adaptive 
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state, This value assertion stems frotn a concern with the person's capacity 
to adapt to J changing environment", (Hunt, 1971: p. 18) This conception 
emphasizes the need for a student-environment match which provides a challenge 
to stimulate growth. Ideally, this could be provided in terms of a classroom 
of which the entirs student population is homon.nooui: in st,3nu of conctntuH 
development". (Hunt, |97h p. 25) 

An earlier version of this work involved an exploratory study, 
intended to 

Obtain some indication of the educational relevance of the model by 
e!<posing classroom groups of the same conceptual stage to the same 
educational environment. Students at a given stage are eKpected to 
share certain common characteristics based on conceptual level. They 
should respond favorably to certain forms of teaching and unfavorably 
to others, even though the environments may not necessarily be optimal 
for progression. ,. in this study we ask, "do thfse classroom stage 
groups differ from one another in expected ways?" and if so, "do 
these differences mal<e any educational sense to "he teachers'" 
( 1971 : p. 26) ■ 

The success of the study lead to further theoretical worl<, and to the 
development of the most recent formulation, the Conceptual Level Matching 
Model, which involves specifying the desired change (educational objectives), 
a conception of the person ( learner characteri sties) , a conception of the 
environment (educational approaches), and a conception of the interactive 
process (theory of Instruction). 

The early results of empirical work using this model have been quite 
interesting, and do suggest that the Conceptual Level of students can be a 
useful guideline for varying classroom groupings and educational approaches. 
Given tha;- some teachers prefer different approaches to others, It also 
suggests the possibility of matching student requirements, based on conceptual 
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levels, and teacher prGdMections, baaed on preferred educational approaches. 
However, the tentat i veness of the empirical results to data, and the relatively 
Gmall samples involved, require that smm caution be exercised in applying this 
particular notion of matching. It remains one of the more Interesting new 
approaches, and should certainly generate a good many empirical studies* 

Implementing the Proposal 

A number of general points about evaluation and its Impact on an 
educational system can be made, In the form of cautions to administrators, 
wiihing to implement an evaluation proposal such as this. The main point 
here Is that evaluation in the context of educational organizations invariably 
has political implications, for the fol lowing reasons (based on Tumin, 1970): 

1, It means different things to different people, because they have 
both different bases of Judgment and aspire to different standards 
of competence. 

2, Consensus agreements become a necessary way of proceeding with 
implementation and this consensus necessarily implies the need 
for some educational growth on the part of participants, and 
suggests that the whole evaluation process in an organization 
needs to develop by evolution rather than by fiat, 

3, The ivahjators need to be open to influence by all parties 
involved not forgetting students, In the planning and imp lamen- 
tation stages particularly* 

Evaluation Is inevitably an exercise of power, and at the very least the 

power to discriminate levels of competence, and at the most the powgr to 

retain or release employees. It thus intrinsically threatening to all 

concerned, although there is some evidence that better than average and 
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non-tenured teachers have somewhat more positive attitudes towards 

evaluation than others. (Wagoner 1, O'Hanlon, n.d) Further, for students, 

evaluation may represent a shift In power in their favor: their partlclpa- 

Mon In rating teachers represents a gain In net power. There Is some 

evidence that this In Itself Is desirable, that 

by giving some power to students, perhaps In the form of choice of 
teacher, or voluntary attendance, or monthly ratings of teacher 
performance one might well inculcate a higher priority for the 
goals of students and of the school system In the minds of teachers 
(Coleman, 1972- p. 46) 

If these cautions are kept In mind, It Is possible that the evaluation 
scheme can In fact contribute positively towards the general health of the 
school division as an organization. There Is soma evidence that the absence 
of control Is relatively undesirable for all concerned. The desirable 
situation is participation In control. (See Tannenbaum, 19523 A study 
of teachers and bureaucracy found, contrary to expectation, that the sense 
of power was higher In highly bureaucratic schools: ••TeBchers In highly 
bureaucratic systems had a significantly higher, not lower, sense of power 
than those in less bureBucratic systems". (Moeller & Charters, 1966: p. 458) 

With regard to more Immediate problems of implementation, two 
related points can be made: the selection, assignment, and retention- 
release decisions regarding teachers will generally be made by a superin- 
tendent or assistant superintendent, and consequently imp lementBtion of 
the program proposed here will depend a good deal on the ability of these 
people to administer penci l-and-paper tests, and to carry out careful 
classroom observation. Fortunately, interaction analysis has been quite 



highly developed, and instructional mate-ials exist which make the skills 
readily teachabis in In-service workshops, (see, for example, Amidon 
& Hunter, 1966) 

The second point concerns in-service programs for teachers. At 
present these are rather poorly developed, primarily because thero Is no 
reasonable theoretical basis for these programs, and consequently they tend 
to be rather haphazard and only marginally effective. The technical skills 
approach to teacher training deveioped at Stanford University (Berliner, 
1969) seems on the surface at least Ideally suited for In-service work. Th 
references here to evaluations used deve I opmenta I ly provide only very 
limited hints as to what the needs of teachers might be, but it does seem 
unlikely that a serious in-servlce program can be developed until after 
some relatively extensive and objective evaluatinn schemes have revealed 
the areas of greatest need. Teacners are not very likely to be aware of 
their deficiencies, although there is littia doubt that student rating 
schemes could help to enlighten them. Thus the implementation of the 
teacher evaluation program suggested here Is seen as a necesssary pre- 
requisite to the development of an adequate in-service program, 

Implementing the proposal for improving teacher assignment requfres 
rather more caution, in the opinion of the writer, than the evaluation- 
based proposals. Consulting teachers on their preferred assignment and 
consulting students on their praferrad teacher both seem reasonable 
practices, ©minently worth trying. However, reducing the discrepancies 
between actual and preferred educational contexts for teachers and students 
will not necessarily result in improved achievement. Careful evaluation 
Of the outcomes would therefore be necessary. The Conceptual Level Model 
of Hunt and colleagues, which emphasizes constructive mismatch, seems likely 
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to be more product iva of desirable student growth. However, this model 
should only be applied under carefuily controNod condltiDns and proferably 
with the assistance of compotent consultants. In essence, all of these 
matching possibflitias require rather careful testing before they can be 
unequivocaMy recommonded , 
Conclusions and Implications 

The approach to improving aggregate af fecti venoss proposed hero 
purports to be timely in that a probable ovorsupply of teachers (for Manitoba, 
<see Husby, 1972; for Saskatchewan, see Scharf, 1972) makes it possible to 
replace release tDachars, and also makes the problem of selecting and 
assigning teachors very much less difficult than previously. Rather than 
having one candidate who seems suitable for the required position, the 
administrator will probably have several candidates, and thus the problem 
of selection and assignmant becomes one of applying more refined processes 
beyond the kind of gross matching which was possible in the past. 

It is also maintained that the approach here Is likely to be effective, 
in that the teacher behaviors and characteristics considered desirable here 
have had in empirical studies In the past positive relationships with good 
attitudes and achievement levels of students; unless evidence to the 
contrary exists, It seems reasonable to expect the positive outcomes to be 
rep I Icated in Manitoba. 

In addition, it Is also mal ntalned that the approach proposed here 
is practical, in that it can be utilized without axtensive retraining of 
supervisors, or heavy expenditures for consultants and data analysis. 
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Unquestionably, some additional prospects for the in-service training of 
administrators are implied by the proposals made here- for examplo, few 
administrators in i-ianitobn hrava had exposuro to verbal interaction analysis 
techniques for evaluating teachQrs, hut this could easily be remedied in a 
school division wishing to adopt these Droposals. 

The proposal can convtjn iontly be summarized in a simpio table: 

TEACHER EVALUATION AND ADMINISTRATIVE DECISION-MAKING: 
DATA NEEDS AND AmiNISTRATOR RESPONSIBILITIES 



Administrators 

Administrative Decisions a Responsible for Collecting 

Evaluation Data Needed Sources of Data and Utll izinq Data 



Be I act ion M , T . A , 



Supari ntendent 



■ (Predictive Data) Vsrabal Facility Test Superintendent 

Development Student Ratings Teacher 

(Formative Data) Interaction Analysis Principal 

Classroom Flehavior Principal 

Retention/Release Interaction Analysis PrincipaU Superintendent 

(oummative Data) Classroom Behavior Principal, Superintendent 

''^■T.A.I, Superintendent 

Additionally, it Is proposed that matching models of three types are 
relevant to tho Improvoment of teacher ©f fcctl vaness ■ one, Improving the 
match between teachers' perfsrencss and training and their tsaching 
assignments; two, Improving the match between teachor and student axpectations 
of role behaviors, and thus reducing role conflict; and three, Improving the 
match between student needs for growth and teacher practices. The first two 
can be implemented to some extant by extending consultation and' choice for 
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teachers and students. 

Close attention by administrators to the processes of teacher 
selection, assignment, development, and rotantl on/re I ease can. It is 
maintained, substantial ly increase the aggregate teaching effectiveness 
of the staff of a school division over a period of some years. Serious 
administrative attempts to improve teaching have not been common in 
recent years. They may well become more popular in response to demands 
for accountability and to the improving supply of teachers. 
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